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ADMINISTRATOR AND INSTRUCTOR COURSE MANAGEMENT 
APPLICATION FOR AN ONLINE EDUCATION COURSE 



CROSS-REFERENCE TO RELATED APPLICATIONS 

The present invention is related to the following co-pending patent 
applications that are assigned to the same assignee as the present invention, 
the subject matter of which are incorporated herein by reference thereto: 

1 . "Virtual University," U.S. Application Ser. No. 09/344,732, filed June 25, 
1999. 

2. "Online Collaborative Apprenticeship," U.S. Application Ser. No. 
09/344,659, filed June 25, 1999. 

3. "Online Virtual Campus," U.S. Application Ser. No. 09/344,754, filed 
June 25, 1999. 

FIELD OF THE INVENTION 

This invention relates to a software learning engine offered online via a 
network of multimedia nodes. More particularly, this invention relates to an 
administrator and instructor course management application for managing 
and monitoring student and instructor activity in an online educational course. 

BACKGROUND 

Traditional campus-based universities are often limited in the number 
of learners they can reach by limitations in their physical resources. 
Classroom and campus constraints generally dictate a relatively small 
consumer base embodied as learners. 

In addition, fixed schedules, a passive learning approach and a 
relatively lengthy academic school year have resulted in the limited education 
models known today based on utilizing a physical campus with one teacher 
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and many learners. One limitation of the prevailing education model is that it 
is only available to people, such as younger students during the first stages of 
their lives, who can dedicate the majority of their time to exclusively studying 
and attending classes. 
5 Known methods of offering distance learning have not been able to 

harness the teaching techniques and material of first tier higher education 
institutions. Existing methods generally require that a learner become self 
taught with periodic assessments by the distance learning institution. 

There is a need for an educational forum that provides interactive 

10 lessons asynchronously such that a learner can participate in a lesson using 

an on-demand schedule and also reap the benefits of an interactive learning 
environment. Such an educational forum should provide for access to large 
numbers of students that may be geographically dispersed while offering all 
students the guidance, feedback and access to resources generally available 

1 5 to traditional campus-based students. There is also a need for instructor tools 

in such an interactive environment to permit instructors to effectively monitor 
and manage students. Additionally, there is a need for administrator tools in 
this interactive environment to facilitate monitoring of instructors and the 
online educational forum as a whole. 

20 BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS 

FIG. 1 illustrates a computer network according to a preferred 
embodiment. 

FIG. 2 is illustrates types of data and applications preferably stored in 
the server side of the computer network of FIG. 1 . 
25 FIG. 3 shows a student graphic user interface for a preferred online 

educational course operative on the computer network of FIG. 1 . 

FIG. 4 is a flow diagram of a method of providing administrators and 
instructors information on students using instructor tool applications. 

FIG. 5 is an online instructor grade book interface generated by an 
30 administrator and instructor tool application according to a preferred 

embodiment. 
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FIG. 6A is an online instructor grade book interface generated by an 
administrator and instructor tool application according to a second preferred 
embodiment. 

FIG. 6B illustrates a graphical grading scale for use in the online grade 
5 book of FIGS. 5 and 6A. 

FIG. 6C illustrates an alternative graphical grading scale for use in the 
online grade book of FIGS. 5 and 6A. 

FIG. 7 is an instructor grade submission template suitable for use with 
the grade book of FIGS. 5 and 6. 
10 FIG. 8 is a student view of the grade book of FIGS. 5 and 6. 

FIG. 9 is a discussion thread rating interface according to a preferred 
embodiment. 

FIG. 10 illustrates a dashboard view containing multiple graphical 
views of information related to a particular student in an online educational 
1 5 course according to a preferred embodiment. 

FIG. 1 1 illustrates a dashboard view containing multiple graphical 
views of information related to students in an entire course section in an 
online educational course according to a preferred embodiment. 

FIG. 12 is an instructor administration screen illustrating instructor 
20 performance information. 

FIG. 13 is an administrator dashboard view summarizing overall 
academic performance in an online educational environment. 

FIG. 14 is an online educational system program selection screen. 
FIG. 15 is an online educational system course selection screen. 
25 FIG. 16 is an interactive chart illustrating completion rates of students 

by section in an online educational system. 

FIG. 17 is an online educational system faculty selection screen. 
FIG. 18 is a faculty data screen for a faculty member selected through 
the faculty selection screen of FIG. 17. 
30 FIG. 19 is a continuation of the faculty data screen of FIG. 18 that 

illustrates faculty performance metrics for the selected faculty member. 
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DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED 
EMBODIMENTS 

In order to enhance an online educational experience and provide for 

effective and efficient student management by online faculty members, a 

5 system and method for providing faculty member tools in an online 

educational environment is described below. Furthermore, administrator tools 

and functionality are disclosed that permit the monitoring and assisting of 

faculty, as well as aiding management of the overall online educational 

environment. In order to provide an example of an appropriate environment 

1 0 within which preferred embodiments of the online faculty member and 

administrator tools may be described, an example of one suitable network and 
course structure is now described. 

Referring to FIG. 1, a preferred online educational network 10 is 
shown. The network 10 includes a server side 12 and a plurality of user 

15 computers 14. Each of the user computers 14 includes a processor, memory, 

a user input device such as a keyboard and/or mouse, a user output device 
such as a video display and may include other user output devices such as 
printers. The user computers 14 may be directly connected to an Internet 
Service Provider (ISP) 15 via a modem or other communication device. The 

20 user computers 14, via an ISP 1 5 or other communication device, may 

communicate with the server side 12 of the network 1 0 over the public 
switched telephone network (PSTN) 17. Alternatively, one or more user 
computers 14 may be connected to the internet via a server already operating 
on an internet backbone. Any of a number of commonly available 

25 communications means may be used by user computers to communicate with 

the internet. 

The server side 12 of the network 10 communicates with the various 
users 14 through routers 16 and switches 18 that direct queries through a 
firewall 20 protecting the server side 12 from unauthorized access. Suitable 
30 routers and switches are Cisco 75xx routers and Cisco 55xx switches 

available from Cisco Systems, Inc. The user computers may interact with the 
server side 12 as if the server side 12 was a single entity in the network 10. 
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The server side 12 of the network 10 may include a central server device or 
may include multiple processing and database sub-networks, such as 
cooperative or redundant processing and/or database servers that can be 
geographically dispersed throughout the network 10. In some 
5 implementations, groups of user computers 14 may communicate with the 

server side 1 2 through a local server 22. The local server 22 may be a proxy 
server or a caching server. 

The server side 12 of the network 10 preferably includes one or more 
databases storing online course programming course materials. A large 

10 variety of course materials may be stored in the databases at the server side 

12. For example, text, audio, video, graphics, animations, and illustrations 
may be included in the materials utilized in the particular online course that a 
student may use for online testing and assessment. A student may access 
the server side 12 of the network 10 using software residing and executed at 

15 the user's computer. The user software may include a generic Hypertext 

Markup Language (HTML) browser, such as Netscape Navigator, Microsoft 
Internet Explorer, or other commonly available Internet web browsers. In 
order to interpret the various multimedia materials utilized in an online 
educational course maintained on the server side 12, the user computers 

20 implement the browser software to interpret HTML and JavaScript commands 

and also preferably use standard browser plug-in, such as QuickTime and 
other commonly available audio and video data interpretation software. 
Although standard Internet connections using browsers that understand 
HTML, JavaScript and other standard Internet programs is preferred, the user 

25 software may also be a proprietary browser and/or other host access 

software. 

Referring to the server side 12 of the network 10, a plurality of 
individual servers 24 may be connected together via an Ethernet connection 
via a local area network 25 such as a 100 Mbps VLAN. Mass storage 
30 devices, such as mass storage device 26, may be in communication with one 

or more of the servers 24 via a hub. Databases stored in the mass storage 
devices 26 may include an Enterprise Resource Planning (ERP) database, a 



-6- 



Customer Relationship Management (CRM) database, and a user profile 
database. The ERP database preferably includes customer and billing 
information and standard payroll and accounting information relevant to the 
online educational system. The CRM database may be a standard CRM 
5 database such as those supported by Oracle Corporation of Redwood 

Shores, California. 

In one embodiment, an online course is preferably stored on the server 
side 12 as a collection of HTML templates programmed to retrieve content 
information from the mass storage device 26 when a user computer 14 

10 accesses a particular page in the online course. In one preferred 

embodiment, the servers 24 may contain online courses programmed using a 
customizable database format such as the database format and programming 
language available from BroadVision, Inc. of Redwood City, California. 
Suitable servers 24 may be E250, E450, and other servers available from Sun 

15 Microsystems, Inc. of Palo Alto, California. The mass storage device 26 may 

be one or more hard disk drive storage devices such as an EMC Symmetrix 
device available from EMC Corporation of Hopkinton, Massachusetts. 

The user computers 14 may be operated by students taking online 
courses, adjunct faculty members (hereinafter instructors) leading the online 

20 courses, advisory faculty members overseeing the performance of a group of 

instructors, or administrators overseeing the online courses and educational 
environment as a whole. Each of the respective types of users preferably 
accesses information in the server side 12 of the network 10 through use of 
user identification and password prompts over the Internet. In one 

25 embodiment, all of the types of users may access the educational information 

and services through a standard Internet web page interfaces. Utilizing the 
above-described structure, students preferably sign up for and take online 
educational courses with other remotely located class mates who are typically 
grouped together in course sections based on course starting date so that, 

30 while each student may proceed at his or her own pace, the students may 

interact and provide some assistance to each other in a group learning 
environment. 
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In a preferred embodiment, the course structure for online courses 
residing on the server side 12 of the network 10 is a problem-based learning 
structure adapted for the online environment. The problem-based learning 
organization of the course materials is designed so that student learning is 
5 driven by learning outcomes linked to authentic projects or problems. 

Students learn on problems having real world relevance using a learning-by- 
doing approach. The "virtual university" embodied in the server side 12 of the 
network 10 provides this type of learning experience by combining the 
resources of traditional learning institutions (e.g., universities), that generally 

1 0 provide courses over a long period of time, with the techniques used in 

training courses, typically more intense, short courses directed towards a 
particular skill or technique. Combining a computerized communication 
network and courses embodied in software in the network allows coursework 
to be accomplished through asynchronous learning. Asynchronous learning 

15 allows a student to access educational material at a time convenient to the 

student while avoiding scheduling conflicts inherent in traditional learning 
institutions. 

The preferred type of courses resulting from the blending of these two 
approaches, along with the problem-based learning technique, are course 

20 suites providing a learning experience, and containing material, not typically 

available in any one place. As shown in FIG. 2, the server side 12 of the 
network 10 contains a course database 40 where each course suite 42 
resides. Each course suite may comprise multiple separate courses 44 that 
cumulatively provide the substance and experience of an entire university 

25 level course, but in discreet logical pieces more suitable for the online 

environment. In one embodiment, the courses 44 consist of templates 45 
created in HTML format using an HTML generator such as Dreamweaver 
available from Macromedia, Inc. of San Francisco, California. One or more 
templates represent the instructions necessary to build each page of the 

30 course 44 requested by a student. The templates 45 may include executable 

software, such as JavaScript commands, and include references to the 
course content data 47, stored in mass storage 26 on the server side 12, 
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necessary to populate the HTML templates. Standard business logic 
programmed in Java or any of a number of commonly available languages 
may be used to provide an interface between any disparate platforms 
maintaining data on the server side 12 using standard common object request 
5 broker architecture (CORBA). 

Students logging on to the Internet from their respective user 
computers 14 to access their particular course 44 will provide identification 
information, such as a user ID and password, to access the course that he or 
she has registered in. If this is the first time a student has accessed or 

1 0 attempted to register for a course, the student will be prompted for appropriate 

biographical and financial information. Once the student has registered for a 
course, an appropriate one of the variety of courses 44 and course suites 42 
stored on the server side 12 of the network 10 are accessed by software code 
operative on the servers 24 to present the appropriate course and materials to 

15 the student. 

Referring to FIG. 2, each student provides biographical data 28 to the 
server side 12 when they register for a course. Biographical data includes 
name, email address, physical address, telephone information, company, and 
position (if employed). Because the online educational services offered in the 

20 network 10 may be accessed by numerous students worldwide, other 

information such as time zone and native language may also be recorded. 
This information on biographic data is preferably stored in a database on the 
mass storage device 26. 

In addition to biographical data, the server side 12 of the network 10 

25 also stores and tracks student information relating directly to the coursework 

and related activities of each student. This course-related student information 
30 includes student status (whether the student is active/inactive), the date 
the particular course began, information as to previous courses the student 
has taken, and the personal schedule of a student as it relates to the 

30 availability of the student to access the online educational environment on a 

timely basis. The course-related student information stored in the server side 
12 also includes information such as whether the student has received the 
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necessary books for the course, whether evaluations for instructors or 
assignments have been completed, the number of complaints logged by the 
student in the present or previous courses, and similar administrative student 
information. Each time a student accesses the online course materials and 
5 resources, a record of the student's access, also referred to as the 

clickstream, is recorded. The student clickstream record may include time, 
date, duration and type of accessed resource data. Other information stored 
in mass storage 26 includes instructor information 49 and course email and 
threaded discussion information 51. 

1 0 An example of a student graphic user interface 48 for one preferred 

embodiment of an online educational course is shown in FIG. 3. The student 
user interface is preferably visible to students on user computers 14 and is 
operative to present student specific course information. In the example of 
FIG. 3, a student's web browser displays information retrieved from the server 

15 side 12 of the network 10 relating to a financial accounting course. 

Information is preferably transmitted in data packet formats that may, in one 
embodiment, conform to the transmission control protocol/Internet protocol 
(TCP/IP). In a TCP/IP implementation, a packet network terminal server in 
the server side 12 is assigned a unique Internet protocol (IP) network 

20 address. TCP/IP switching equipment within the network can direct a TCP/IP 

packet to an intended recipient based on the packets' destination IP address. 
Other implementations may use other networking protocols and packet 
formats. 

As mentioned above, the information passed to the user computer 14 
25 is preferably assembled from a collection of HTML templates stored on one or 

more servers operating database software such as BroadVision database 
software. Information and data for the particular course page represented by 
the HTML template is drawn from the mass storage device. The resulting 
stream of information, generated by populating the HTML template in the 
30 server side of the network with the data required by the template, is sent to 

the user computer. The browser software interprets the HTML commands 
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received to arrange the accompanying course data so that the student sees a 
screen such as shown in FIG. 3 on her user computer. 

Additionally, each time a student logs on to the online educational 
service, the server side 12 of the network 10 records the students presence. 
5 All of the student's subsequent activities are also recorded and catalogued by 

the databases residing on the server side 12 so that a complete picture of the 
student's activities is recorded. Each student's activities may be recorded 
based on the student's login information or other mechanisms, such as 
cookies, such that every selection and activity by the student is recorded. 

10 Referring again to FIG. 3, the interface 48 displayed on a user computer 

display to a student taking a course provides access to multiple resources. In 
one embodiment, the interface 48 may consist of a course resource 
navigation bar 50, an upper navigation bar 52, and an information display 
region 54. The image displayed in the display region 54 to the student who 

1 5 selects the course guide link 56 shows an image of the portions of the 

selected course. 

When the user selects the course guide link 56, the user triggers a 
query for information located at an address identified by, for example, a 
Universal Resource Locator (URL) containing an address in hypertext transfer 

20 protocol (HTTP) associated with the course information in the servers on the 

server side 12 of the network 10. Information at the address is then retrieved, 
assembled and sent to the user computer 14 for presentation in the display 
region 54. Each of the links in the course resource navigation bar 50 on the 
interface 48 sends a request for information to the server side of the network 

25 and causes information to be retrieved. Examples of materials that a student 

may access include introductory materials such as the course guide 56, 
project information including a list of project tasks 58 that the student is 
required to complete in order to complete the course, resources and tools 60 
related to the task presented to the student in the course, and numerous 

30 research modules 62 that summarize key points of knowledge that a student 

will need to know to complete the tasks. The research modules may provide 
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background information on various topics, practice exercises and links to 
multimedia (video, audio, etc.) explanations of methods and terminology. 

Students taking online courses may also link to, in a preferred 
embodiment, course discussion threads. The course discussion thread is 
5 preferably an online bulletin board maintained in the server side 12 of the 

network and accessible only to students of a particular course section and 
their instructor. Course discussion threads are focused on questions and 
issues directly related to the course. Both students and instructors may post 
messages to the course discussion thread. The course discussion thread 

10 may be accessed through a top navigation bar 52 that provides a discussion 

link 64. In one embodiment, the course discussion thread is maintained in 
one or more servers on the server side 12 of the network utilizing Lotus 
Domino software. Other links, such as a library resources link 66, a student 
home link 68, and a technical support link 70 are accessible on the upper 

15 navigation bar 52. 

According to a preferred embodiment, an administrator and instructor 
tool application 80 (FIG. 2) provides authorized administrators and instructors 
(who may be advisory faculty or general instructors) for the online educational 
network 10 with the ability to manage and monitor student activity. In the 

20 following discussion, examples of instructor-level functionality are first 

discussed. Preferably, all authorized users, other than students, have access 
to instructor-level functionality. In the embodiment described below, an 
instructor is identified as utilizing certain functionality. Advisory faculty and 
administrators for the online educational network also have access to all of 

25 the instructor-level functionality. Each of the instructors, advisory faculty, and 

administrators is identified by the online educational network through her 
respective logon and password information. Any of a number of known 
authentication and security techniques may be used for this purpose. In one 
embodiment, the user interface may display a complete set of information and 

30 functionality access mechanisms to all users and deny access to users on a 

request-by-request basis. In another embodiment, the user interface may be 
tailored to display only those information and functionality access 
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mechanisms appropriate to that particular user. Although the embodiment 
described below provides a hierarchy where instructors have a first level of 
access, advisory faculty have a second level of access that includes the first 
level, and administrators have the highest level of access (including 
5 everything available at the first and second levels), other embodiments may 

provide additional layers of information and functionality access, or access to 
various pieces within a certain level. 

Referring to FIG. 4, an instructor provides logon information, for 
example a user ID and a password, to the server side 12 through a user 

10 computer 14 (at 82). The server side 12 authenticates the instructor's 

information (at 84) and will receive and act on a student information query (at 
86) if the instructor is an authorized instructor registered with the online 
education network. In response to the instructor query, the server side 
accesses the administrator and instructor tools 80 and transmits student 

1 5 management tool selections to the instructor (at 88). When an administrator 

and instructor tool selection is made by the instructor, the appropriate student 
information is retrieved for the instructor from the student biographical 
information database 28, course related student information database 30, and 
any other databases maintaining student information. Preferably, the 

20 instructor only is permitted access to student information for students in her 

course sections. Specific types of information and instructor tools are 
described in greater detail below. The functionality of each of the 
administrator and instructor tools 80 available on the server side 12 of the 
network may be implemented in software code operative to retrieve and 

25 populate HTML templates with requested data in the same fashion as 

described above with respect to student access of the online educational 
courses 44. 

Referring to FIG. 5, an embodiment of one interactive instructor tool, 
specifically a grade book screen 100 provides instructors for the online 
30 educational courses current information on grades assigned to students in 

their section for various tasks required of the course. As described above, 
only students assigned to a particular instructor are accessible by the 
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instructor. If an instructor is teaching more than one section of a course, the 
instructor may provide information to indicate which section she is interested 
in and receive displayed information for that particular section. As shown in 
FIG. 5, an embodiment of an online grade book includes a listing of student 
5 names 102 in a table format associated with grade entries 104 for those tasks 

graded and previously posted. Although the embodiment of FIG. 5 indicates 
letter grades assigned to each graded project, other grading scales may be 
implemented. 

Students preferably submit completed tasks in an electronic format, for 

10 example via email, to the instructor so that a database in the server side 12 of 

the network 10 may then store the assignment and create a link to its location 
for the instructor the next time the instructor accesses the grade book. In one 
embodiment, students may attach a completed assignment to an assignment 
inbox via a student grade book view screen displayed on the student 

15 computer. The student view 131 of the grade book, as shown in FIG. 8, 

provides a view of the grade 133 assigned for each assignment 135 
submitted, along with the date submitted 137 and date graded 139. The 
student sees a submit button 141 for each assignment remaining. Each 
submit button 141 preferably automatically links the student to an assignment 

20 submission form pre-populated with relevant information such as student 

name, course name, section identifier and task identifier. The assignment 
attached to the assignment submission form is routed to a database, such as 
a CRM database as described above, when the student submits the 
completed assignment. A time stamp is automaticaliy generated at the 

25 database upon receipt of the assignment. Additionally, the data on the pre- 

populated form may be used to cross-reference the assignment to the section, 
course, instructor and other parameters for later retrieval and use. 

Referring to the embodiment of FIG. 6A, an alternate grade book 
display format 120 is illustrated with a grade scale made up of non-traditional 

30 letter indicators 122, such as M, P, I and N (for Mastery, Passing, Needs 

Improvement and Incomplete, respectively). In addition to the letter grade, 
the grade book display may include an assignment attempt indicator 124 
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denoting a student's first attempt at a homework assignment with a single 
indicator marker and a second attempt at a homework attempt with multiple 
markers. Alternate embodiments of graphical grade indicators 127, 129 are 
illustrated in FIGS. 6B and 6C. 
5 In order to assist instructors in prioritizing which student assignments 

require attention, the grade book administrator and instructor tool preferably 
generates an elapsed time indicator 126 indicating the time elapsed from 
when each student submitted a homework assignment. This elapsed time 
may be generated by the instructor grade book tool on the server side of the 

1 0 network 1 0 by comparing the time stamp of the received homework to the 

time that the instructor logs on to the network. As shown in FIG. 6A, the 
elapsed time markers 126 may include text and/or graphics. Preferably, the 
visibility of the marker 126 changes to indicate greater urgency as more time 
elapses. The change in the marker 126 may be implemented as an increased 

15 size, a pixel blinking mechanism, or other visual indicator. Additionally, the 

markers 126 may be set to change size or other features after predetermined 
time increments. The instructor grade book tool updates the elapsed time 
markers displayed to an instructor each time the instructor logs on to the 
network, or in an alternative environment, may be automatically refreshed 

20 through instructing the instructor's browser software to automatically refresh 

the link. Similar to grade button 109 shown in the embodiment of FIG. 5, the 
display 120 of FIG. 6A provides a visible, ungraded student assignment link 
128 allowing the instructor ready access to a particular student's ungraded 
assignment. Preferably, the instructor grade book display incorporates 

25 redundant perceptual queues, such as combinations of shape, color, blinking 

features or icon movement, etc., to enhance administrator or instructor 
comprehension of grade related information. 

When an instructor views the grade book display screen and decides 
on a student assignment to grade, the instructor clicks on the button or other 

30 indicator to retrieve the student's assignment from the database side of the 

network. The instructor may download an assignment to view immediately, or 
the instructor may download and print the assignment to grade at a later time. 
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As shown in FIG. 7, the link selected by the instructor provides the instructor 
with a grade submission template 130 identifying the student assignment 132 
and containing an comments 134 provided by the student to explain issues 
related to the assignment. The grade submission template 130 provides the 
5 instructor with a text box 136 within which to generate a message to the 

students with comments on the assignment. After reviewing the assignment 
through an automated link in the grade book, the instructor accesses a grade 
menu 138, such as a pull down menu listing letter or symbol grade 
designations, and selects the appropriate grade. When the instructor finishes 

10 grading, she may click on the save button 140 to transmit the grade and any 

comments back to the database in the server side of the network. In other 
preferred embodiments, the grade submission template and/or the grade book 
include links to previously graded assignments for the student or other 
student's graded assignments along with the option of displaying different 

15 assignments alongside one another. The grade menu 138 of the grade 

submission template may include a collection of prepared comments or hints 
for the instructor to access and paste into the text box 1 36 when grading 
assignments. Once a grade is entered, the student can view the grade in a 
student view 131 of the grade book (See FIG. 8). 

20 As discussed above, one useful teaching and communication tool 

provided in a preferred embodiment of the online educational courses is the 
guided discussion thread. The discussion threads operate in an online 
bulletin board environment to permit students, and the instructor, to read and 
post messages on issues directly related to the course. Preferably, the 

25 instructor is provided access to the discussion thread through the instruction 

interface tools 80. Referring again to FIG. 5, an instructor may view a 
discussion by selecting a link 106 on a navigation bar 108 displayed to the 
instructor to retrieve a listing of postings to the discussion by topic and date. 
In order to assist the instructor in assessing student participation in discussion 

30 threads, a discussion thread rating interface 150, such as illustrated in FIG. 8, 

permits the instructor to selectively open and attach ratings to various 
messages. Activating the discussion thread link 106 (FIG. 5) preferably 
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triggers the administrator and instructor tool module 80 to assemble the 
discussion thread rating interface 150 and draw in the appropriate posting 
references into the first window 152. in one embodiment, a first window 152 
is presented to an instructor displaying a scrollable list of the postings in the 
5 discussion thread and a second window 154 displays a selected one of the 

messages in the scrollable list 152. Alternatively, the instructor may access 
the discussion thread through a link 64 (FIG. 3) on a navigation bar 52 on the 
instructor's display. 

An instructor preferably uses the discussion thread tool interface 150 to 

10 review and append a rating to student postings. The instructor appends a 

rating to a posting by selecting a rating 156 provided in a pull down menu 158 
in the discussion thread interface 150. Preferably, the assigned rating is then 
indicated next to the student posting in the form of a graphic or icon 160 in the 
listing of postings viewed in the lower window 1 52 of the instructor discussion 

15 thread interface 150. Although the ratings are displayed to the instructor, 

students preferably do not see any indication of the assigned rating. A 
discussion thread navigation bar 162 on the discussion thread instructor 
interface 150 contains buttons operative to sort postings by rating 164, dates 
166, author 168 and topic 170. Other discussion thread navigation functions, 

20 such as a button 172 to display all messages directed to one topic or search 

button 174 to look for messages based on customized search criteria are also 
provided in the discussion thread navigation menu 162. In one embodiment, 
student postings may count toward a task grade in a particular online course. 
The ratings provided by an instructor may be tallied and used to efficiently and 

25 automatically generate a grade for each student relating to postings. For 

example, the instructor may count up the number of "excellent" or "good" 
comments that she has given to a student and provide a indication that the 
student rates a separate commendation (e.g. a statement of "with distinction") 
in addition to the overall course grade. Conversely, the ratings assigned to 

30 the discussion threads may be a tool to quickly identify inappropriate postings, 

for example a posting that constitutes sexual harassment or some other type 



of inappropriate behavior, so that students may be warned or removed from 
the course. 

As part of the administrator and instructor tool set 80 available to 
instructors for managing and monitoring student activity (and to administrators 
to monitor student and instructor activity), instructors preferably may request a 
variety of information on students in their sections, student performance, and 
their own efficiency as instructors. Not only does the instructor tool set 
provide access to up-to-date information, it preferably generates graphical or 
tabular representations of the data discreetly or collectively. An instructor 
may request individual or section data in this manner. For example, as 
illustrated in FIG. 10, an instructor can quickly access a particular student's 
performance using a dashboard view screen 180 generated from data stored 
in the server side of the network as requested by commands in the 
administrator and instructor tool set 80. This data is accumulated from 
submissions from the student (e.g. biographical data provided during course 
registration) and by the student's activity in the online course. 

A first section 182 of the dashboard view screen preferably presents 
biographic data for the student, such as name, address and contact 
information. A second section of the dashboard view may be a graphical 
representation of course activities. The course activity information may be 
generated from a number of clicks, or a number of times a link is accessed by 
a student, for activities directly related to the online course. Course activities 
may include viewing research modules related to the course, accessing the 
tasks in the course, posting or viewing material in the discussion threads, or 
viewing the resources and tools provided to assist the student in the course. 
In one embodiment, the second section 184 of the dashboard view 180 may 
illustrate the course activity in terms of a bar chart 186 having a separate bar 
for each activity tracked and extending along an axis listing the number of 
clicks by the student into that course activity. In alternative embodiments, 
overlaid linear graphs may be generated having an X-axis illustrating time 
(week number of the course) and a Y-axis indicating the number of clicks. In 
a third portion 186 of the dashboard profile, a similar graphical representation 
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of out-of-course activities may be presented. The out-of-course activities may 
include postings to threaded discussions related to other online educational 
issues posted by the host system, the number of times the online library 
resources have been accessed, clicks into chat room discussions, and other 
5 activities indirectly related to the course. Another section 188 of the 

dashboard view 180 may contain a link 190 to automated email forms 
providing a pre-addressed form for the instructor to directly email the student. 
In other links such as a threaded discussion link 192 that may, for example, 
access recent posting by the students to allow the instructor to post a note in 

1 0 the discussion thread to the student. 

As illustrated in FIG. 1 1 , a similar dashboard view 200 may be 
accessed through the instructor interface to invoke the administrator and 
instructor tools 80 to retrieve and generate class profile information. A 
student demographic section 202 may occupy one section of the class profile 

15 dashboard view 200. The student demographic section 202 includes course 

and student summary information regarding the number of students, the 
number of students without the recommended course books, a listing of 
complaints or compliments logged and related information. The student 
demographic data section 202 may also be configured to display graphical 

20 status information for the section including a bar chart 203 of student 

assignment submissions graphically illustrating the number of students having 
turned in the assignment for each of a preset number of assignments in the 
online course. An course activity section 204 and an out-of-course activity 
section 206, similar to the sections displaying of these pieces of information 

25 for an individual student, may be included. A class communication section 

208 preferably provides for a link 210 to automatic email templates that are 
pre-addressed to students in the instructor's section and a link 212 to the 
threaded discussion for the section. Each of the sections providing students 
course section information to an instructor preferably allows the instructor to 

30 zoom in and display the section on the entire screen presented to the 

instructor. Alternatively, the instructor interface preferably permits the 
instructor to select links in navigation bars on the screen to directly go to a 
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specific type of information illustrated in the composite view. For each section 
of the composite display, or if only a particular one of the student information 
displays is shown, a student selection mechanism, such as the pull down 
menu 205 shown adjacent the course activity section 204, may be provided 
5 for ease of instructor access to a particular student. 

Another administrator and instructor tool operative on the server side 
12 to assist instructors is an instructor reminder mechanism that can be 
manifested in the form of a link on an instructor interface to students needing 
particular attention. Referring again to FIG. 5, the instructor interface 

1 0 generated by the programming operative on a processor in the server side of 

the network can provide a link 107 identified as "students needing attention" 
that retrieves a list of students having any of a number of difficulties or 
performance concerns. For example, the administrator and instructor tool 
programming can automatically generate a list of students who have not 

15 accessed the online course materials in more than a recommended period of 

time. The administrator and instructor tool can also compile information on 
students needing attention due to the instructor's tardiness in responding to 
course duties. For example, the instructor reminder mechanism can compile 
and display to the instructor outstanding email inquiries from students or 

20 ungraded assignment reminders that need attention. 

In addition to the student performance tracking capabilities provided to 
instructors, the administrator and instructor tools provide instructors with self- 
evaluation information. Preferably, each instructor submits a time log to the 
online education network on a regular basis. The time log may be accessed 

25 with, and information entered through, an instructor administration screen set 

selectable by the instructor through an administration navigation bar 220 such 
as shown in FIG. 12. The time log link 222 preferably calls up a time log entry 
screen prepopulated with the instructor's name, course section and time of 
last log entry. The instructor then enters in the time spent on various key 

30 tasks such as monitoring discussion threads, grading, reviewing and 

responding to student emails, handling technical problems and other time 
dedicated to the online education system. After entering the various times in 
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the form, the instructor may submit the form using an automated link that 
sends the information to a database, such as a CRM database, maintained on 
the server side 10 of the system. 

Referring again to FIG. 12, the instructor may access her performance 
5 information using a performance link 224 on the navigation bar 220 of the 

administration display 226. The instructor may select from a performance 
period menu 228 and a peer group menu 230 to create a textual or graphical 
comparison chart 234 showing instructor performance criteria. The 
performance period menu 228 may present selections of performance since 

10 previous review, performance over past week or month, or performance since 

the start of the course. The peer group selection 230 may include comparison 
with other instructors in a Net, defined as the group of instructors supervised 
by a particular advisory faculty member, other instructors teaching the same 
course, or all other instructors active in the online education system 10. The 

1 5 instructor may then select to regenerate the displayed information according 

to the selected performance period and peer group using a redraw button 232 
displayed on the instructor's computer screen. 

The graphical comparison chart generated for the instructor may 
include separate bar charts for student completion rate, average turnaround 

20 time for grading assignments, the average time a student needs to complete a 

course, the average evaluation received from students and the quality of 
interaction with students. Preferably, all of the information is automatically 
gathered from the databases in the server side 12 based on instructor log 
entries, student evaluations, and automatically date stamped information 

25 maintained in the system. Any of a number of commonly available tools for 

reporting and presenting data, such as those available from BroadBase, 
BroadVision or Oracle, may be utilized and executed on the server side of the 
system to generate averages or percentages that the instructor's computer 
receive in the form of textual or graphical information. The instructor may also 

30 directly email the appropriate advisor, faculty member or an administrator 

using the same administration display 226 with any questions or comments 
relating to instructor performance using advisory faculty or administrator email 
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links 236, 238 that automatically generate pre-addressed email forms, or 
email forms with menu selectable addresses. 

FIG. 13 illustrates an administrator dashboard view of overall academic 
performance 240 accessible by administrators of the online educational 
5 network and assembled by the administrator and instructor tool set. The 

administrator dashboard view is automatically updated from data maintained 
in the server side of the network each time the administrator dashboard view 
240 is accessed. The dashboard view indicates the view level 242 
represented by the data. In FIG. 13, the view level 242 is indicated by a 

10 graphic as the highest (e.g. online educational network as a whole) such that 

the data is an overall picture of academic performance for the online 
educational network. Other selectable levels of view include a program view 
(e.g. all students of an MBA program), course view, faculty view and student 
view. The data presented to an administrator steadily narrows in focus from 

1 5 the top of the view level 242, where data relevant to the entire online 

institution is presented, to the individual student level where the administrator 
may focus on just one student. 

In the online educational network view, referred to as CARDEAN in 
FIG. 13, the dashboard view generates an enrollment chart 244, a course 

20 completion rate chart 246, a student withdrawal rate chart 248, an assignment 

turnaround time chart 250 a student return rate chart 252, and a time to 
completion chart 254. All these charts are preferably positioned on the same 
computer display page so that the charts are viewable simultaneously on the 
screen or by scrolling down on the same screen. Although the information is 

25 shown in the form of bar charts in FIG. 1 3, any of a number of known 

graphical formats may be used for displaying the information in the dashboard 
view screen 240. In other embodiments, the data for a given chart may be 
displayed in alphanumeric form. Each of the charts may include a time scale 
selection 256 that will manipulate or average data stored on the server side 

30 for a desired time period. The administrator and instructor tools may be 

programmed to provide fixed time scale selections such as the 1 day, 5 day, 3 
month, etc. shown, or a customized selection where an administrator may 
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enter a desired time scale. In one embodiment, the same default time scale 
is automatically applied to each of the different information charts when an 
administrator initially calls up the dashboard view 240 onto her screen. 

The enrollment chart 244 includes the percentage of capacity or goal 
5 for enrollment for a given time period. A completion rate chart 246 provides 

the percentage of students completing each of a selected number of courses 
(e.g., finance, accounting, marketing). The withdrawal rate chart presents 
percentage of students voluntarily withdrawing from each of a selected group 
of courses. The assignment turnaround time chart 250 presents 

1 0 administrators with the average overall time it takes instructors for each of a 

selected group of online courses to review and grade assignments, exams, or 
other student course deliverables. The return rate chart 252 represents the 
percentage of students, who have previously enrolled in courses in the online 
educational system, returning to take additional online courses. A time to 

1 5 completion chart 254 on the administrator dashboard display 240 presents the 

average length of time student have taken to complete each of a selected 
group of online courses. Each of the charts on the administrator dashboard 
240 may also include a superimposed graphic indicator 258 such as 
illustrated in the return rate chart 252. The superimposed graphic indicator 

20 258 may be selected to illustrate a desired goal for the data displayed or it 

may show a historical trend for the more recent data displayed in the chart. 
For example, the indicator 258 in the return rate chart may show the average 
November, December and January return rates for the past 10 years, while 
the bar charts indicate the return rate for November, December and January 

25 of the present year. 

The view level indicator 242 allows the administrator to retrieve a 
degree program (e.g., MBA, MSIT, etc.) screen 260. In one embodiment, the 
degree listing screen 260 includes a listing of degree programs 262, course 
suites 264 required for the degree program 262, and any discrete courses 264 

30 within the course suites as shown in FIG. 14. Each one of the programs, 

course suites and discrete courses preferably include a link that automatically 
queries the databases on the server side of the system for academic 
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performance data tailored to the selected program, course suite, or discrete 
course. Thus the administrator dashboard view 240 may be generated from 
the program selection level, or the course selection level, to provide statistical 
information relevant to the program or course in the same manner as was 
5 described for the entire online educational system in FIG. 13. 

Similarly, administrators can select the view level 242 to provide 
specific course information. As shown in FIG. 15, a course selection menu 
268 lists all of the courses offered by the online educational system. Each 
course selection link 270 is configured to respond to a user selection to query 

10 the database on the server side of the system and return relevant information 

concerning the selected course. For example, one information format 
provided by the administrator and instructor tools at the course view level is a 
completion by team chart 272, as shown in FIG. 16. The completion by team 
chart presents the percentage of students completing the selected course for 

1 5 each course section 274. The completion by team chart 272 includes student 

section links 276 and links 278 to faculty for each section. These links 276, 
278 permit quick and convenient navigation capabilities for the administrator. 

When a faculty or student view is selected in the view level area 242 of 
the screen, an administrator may track down a specific faculty member or 

20 student. FIG. 17 illustrates a faculty selection screen 280 containing faculty 

links 282 to all data for each listed faculty member/instructor. If an 
administrator or advisory faculty member needs to review a particular faculty 
member, the administrator or advisory faculty member selects the desired 
faculty member link 282 and is presented with that faculty member's screen 

25 283 listing personal data listing 284 and history with the online educational 

system 286 as seen in FIGS. 18-19. The personal data listing 284 may 
include contact information 286, educational background 290 and so on. The 
history 286 section preferably includes information on when the faculty 
member was trained 292, a list of courses previously or presently taught 294, 

30 evaluation status 296, a list of any student complaints 298, and an automatic 

messaging link 300 that will present a pre-addressed email or messaging 
template for the administrator to contact the faculty member. As with the 
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other screens generated by the administrator and instructor tool, all 
information on the faculty member is automatically updated each time a user 
accesses the screen. 

As shown in FIG. 19, a team performance dashboard view 302 for the 
5 course section(s) taught by the faculty member is also presented when the 

particular faculty member's screen 283 is accessed. Examples of data 
presented to the administrator or advisory faculty member include a 
completion rate chart 304, a withdrawal chart 306, and a return rate chart 308 
for tracking the percentage of students completing or withdrawing from 

1 0 courses taught by the faculty member, as well as tracking how many students 

return to take other courses offered at the online educational system after 
taking a course from the selected faculty member. Information charts for the 
faculty member's assignment turnaround time 310, the number of complaints 
received 312 and the time students take to complete courses taken from the 

15 selected faculty member 314 are also listed. 

From the foregoing, an administrator and instructor application tool 
defining a system and method for managing and monitoring students and 
faculty in an online educational course has been described. Using the system 
and method described above, an instructor can efficiently review individual 

20 student performance or course section performance through instructor 

interfaces generated by administrator and instructor tools maintained on the 
servers of the online educational course network. With the ready access to 
student activity data, biographical data and communications media, such as 
email and discussion threads, an instructor using the student managing and 

25 monitoring tools can effectively assess student difficulties and quickly react to 

student needs. Typical instructor tasks, such as grading assignments, are 
also facilitated by the administrator and instructor tools described above. 
Similarly, administrators and advisory faculty can manage and monitor 
instructors and students the faculty application tool to gather and present data 

30 on the online educational system as a whole, or to easily focus in a specific 

faculty member or student and address specific needs of that student or 
faculty member. 
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It is intended that the foregoing detailed description be regarded as 
illustrative rather than limiting, and that it be understood that the following 
claims, including all equivalents, are intended to define the scope of this 
invention. 



